Interactive Perception: Closing the Gap Between Action and Perception

We introduce Interactive Perception as a new perceptual paradigm for autonomous robotics in unstructured environments. Interactive perception augments the process of perception with physical interactions, thus integrating robotics and computer vision. By integrating interactions into the perceptual process, it is possible to manipulate the environment so as to uncover information relevant for the robust and reliable execution of a task. Examples of such interactions include the removal of obstructions or object repositioning to improve lighting conditions. More importantly, forceful interaction can uncover perceptual information that would otherwise be imperceivable. In this paper, we begin to explore the potential of the interactive perception paradigm. We present an interactive perceptual primitive that extracts kinematic models from objects in the environment. Many objects in everyday environments, such as doors, drawers, and hand tools, contain inherent kinematic degrees of freedom. Knowledge of these degrees of freedom is required to use the objects in their intended manner. We demonstrate how a robot is capable of extracting a kinematic model from a variety of tools, using very simple algorithms. We then show how the robot can use the resulting kinematic model to operate the tool. The simplicity of these algorithms and their effectiveness in our experiments indicate that Interactive Perception is a promising perceptual paradigm for autonomous robotics.